Evaluating Answer Validation in Multi-stream Question Answering
نویسندگان
چکیده
We follow the opinion that Question Answering (QA) performance can be improved by combining different systems. Thus, we planned an evaluation oriented to promote the specialization and further collaboration between QA systems. This multistream QA requires to develop the modules able to select the proper stream according to the question and the candidate answers provided. We describe here the evaluation framework we have developed with special focus on the evaluation measures and the study of their behavior in a comparative evaluation.
منابع مشابه
Answer Selection in a Multi-stream Open Domain Question Answering System
Question answering systems aim to meet users’ information needs by returning exact answers in response to a question. Traditional open domain question answering systems are built around a single pipeline architecture. In an attempt to exploit multiple resources as well as multiple answering strategies, systems based on a multi-stream architecture have recently been introduced. Such systems face...
متن کاملTowards Multi-Stream Question Answering Using Answer Validation
Motivated by the continuous growth of the Web in the number of sites and users, several search engines attempt to extend their traditional functionality by incorporating question answering (QA) facilities. This extension seems natural but it is not straightforward since current QA systems still achieve poor performance rates for languages other than English. Based on the fact that retrieval eff...
متن کاملMaking Stone Soup: Evaluating a Recall-Oriented Multi-stream Question Answering System for Dutch
We describe the participation of the University of Amsterdam in the Question Answering track at CLEF 2004. We took part in the monolingual Dutch task and, for the first time, also in the bilingual English to Dutch task. This year’s system is a further elaboration and refinement of the multi-stream architecture we introduced last year, extended with improved candidate answer re-ranking and filte...
متن کاملLogical Validation, Answer Merging and Witness Selection - A Study in Multi-Stream Question Answering
The paper presents an approach to multi-stream question answering (QA) using deep semantic parsing and logical validation for filtering answer candidates. A robust entailment check is accomplished by embedding the prover in a relaxation loop. Fallback strategies ensure a graceful degradation of performance in the case of parsing problems. The logical validity score is complemented by false-posi...
متن کاملINAOE at QA@CLEF 2008: Evaluating Answer Validation in Spanish Question Answering
This paper introduces the new INAOE’s answer validation method. This method is based on supervised learning approach that uses a set of attributes that capture some lexical-syntactic relations among the question, the answer and the given support text. In addition, the paper describes the evaluation of the proposed method at both the Spanish Answer validation Exercise (AVE 2008) and the Spanish ...
متن کامل